Learning to Navigate the Energy Landscape

In this paper, we present a novel and efficient architecture for addressing computer vision problems that use `Analysis by Synthesis'. Analysis by synthesis involves the minimization of the reconstruction error which is typically a non-convex function of the latent target variables. State-of-the-art methods adopt a hybrid scheme where discriminatively trained predictors like Random Forests or Convolutional Neural Networks are used to initialize local search algorithms. While these methods have been shown to produce promising results, they often get stuck in local optima. Our method goes beyond the conventional hybrid architecture by not only proposing multiple accurate initial solutions but by also defining a navigational structure over the solution space that can be used for extremely efficient gradient-free local search. We demonstrate the efficacy of our approach on the challenging problem of RGB Camera Relocalization. To make the RGB camera relocalization problem particularly challenging, we introduce a new dataset of 3D environments which are significantly larger than those found in other publicly-available datasets. Our experiments reveal that the proposed method is able to achieve state-of-the-art camera relocalization results. We also demonstrate the generalizability of our approach on Hand Pose Estimation and Image Retrieval tasks

OutCast: Outdoor Single-image Relighting with Cast Shadows

We propose a relighting method for outdoor images. Our method mainly focuses on predicting cast shadows in arbitrary novel lighting directions from a single image while also accounting for shading and global effects such the sun light color and clouds. Previous solutions for this problem rely on reconstructing occluder geometry, e.g. using multi-view stereo, which requires many images of the scene. Instead, in this work we make use of a noisy off-the-shelf single-image depth map estimation as a source of geometry. Whilst this can be a good guide for some lighting effects, the resulting depth map quality is insufficient for directly ray-tracing the shadows. Addressing this, we propose a learned image space ray-marching layer that converts the approximate depth map into a deep 3D representation that is fused into occlusion queries using a learned traversal. Our proposed method achieves, for the first time, state-of-the-art relighting results, with only a single image as input. For supplementary material visit our project page at: https://dgriffiths.uk/outcast.Comment: Eurographics 2022 - Accepte

Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade

Camera pose estimation is an important problem in computer vision. Common techniques either match the current image against keyframes with known poses, directly regress the pose, or establish correspondences between keypoints in the image and points in the scene to estimate the pose. In recent years, regression forests have become a popular alternative to establish such correspondences. They achieve accurate results, but have traditionally needed to be trained offline on the target scene, preventing relocalisation in new environments. Recently, we showed how to circumvent this limitation by adapting a pre-trained forest to a new scene on the fly. The adapted forests achieved relocalisation performance that was on par with that of offline forests, and our approach was able to estimate the camera pose in close to real time. In this paper, we present an extension of this work that achieves significantly better relocalisation performance whilst running fully in real time. To achieve this, we make several changes to the original approach: (i) instead of accepting the camera pose hypothesis without question, we make it possible to score the final few hypotheses using a geometric approach and select the most promising; (ii) we chain several instantiations of our relocaliser together in a cascade, allowing us to try faster but less accurate relocalisation first, only falling back to slower, more accurate relocalisation as necessary; and (iii) we tune the parameters of our cascade to achieve effective overall performance. These changes allow us to significantly improve upon the performance our original state-of-the-art method was able to achieve on the well-known 7-Scenes and Stanford 4 Scenes benchmarks. As additional contributions, we present a way of visualising the internal behaviour of our forests and show how to entirely circumvent the need to pre-train a forest on a generic scene.Comment: Tommaso Cavallari, Stuart Golodetz, Nicholas Lord and Julien Valentin assert joint first authorshi

Free-viewpoint Indoor Neural Relighting from Multi-view Stereo

We introduce a neural relighting algorithm for captured indoors scenes, that allows interactive free-viewpoint navigation. Our method allows illumination to be changed synthetically, while coherently rendering cast shadows and complex glossy materials. We start with multiple images of the scene and a 3D mesh obtained by multi-view stereo (MVS) reconstruction. We assume that lighting is well-explained as the sum of a view-independent diffuse component and a view-dependent glossy term concentrated around the mirror reflection direction. We design a convolutional network around input feature maps that facilitate learning of an implicit representation of scene materials and illumination, enabling both relighting and free-viewpoint navigation. We generate these input maps by exploiting the best elements of both image-based and physically-based rendering. We sample the input views to estimate diffuse scene irradiance, and compute the new illumination caused by user-specified light sources using path tracing. To facilitate the network's understanding of materials and synthesize plausible glossy reflections, we reproject the views and compute mirror images. We train the network on a synthetic dataset where each scene is also reconstructed with MVS. We show results of our algorithm relighting real indoor scenes and performing free-viewpoint navigation with complex and realistic glossy reflections, which so far remained out of reach for view-synthesis techniques

How chains and rings affect the dynamic magnetic susceptibility of a highly clustered ferrofluid

The dynamic magnetic susceptibility, χ(ω), of a model ferrofluid at a very low concentration (volume fraction, approximately 0.05%), and with a range of dipolar coupling constants (1≤λ≤8), is examined using Brownian dynamics simulations. With increasing λ, the structural motifs in the system change from unclustered particles, through chains, to rings. This gives rise to a nonmonotonic dependence of the static susceptibility χ(0) on λ and qualitative changes to the frequency spectrum. The behavior of χ(0) is already understood, and the simulation results are compared to an existing theory. The single-particle rotational dynamics are characterized by the Brownian time, τB, which depends on the particle size, carrier-liquid viscosity, and temperature. With λ≤5.5, the imaginary part of the spectrum, χ′′(ω), shows a single peak near ω∼τB-1, characteristic of single particles. With λ≥5.75, the spectrum is dominated by the low-frequency response of chains. With λ≥7, new features appear at high frequency, which correspond to intracluster motions of dipoles within chains and rings. The peak frequency corresponding to these intracluster motions can be computed accurately using a simple theory. © 2021 American Physical Society.A.O.I. gratefully acknowledges research funding from the Ministry of Science and Higher Education of the Russian Federation (Ural Mathematical Center Project No. 075-02-2021-1387)

Effects of nanoparticle heating on the structure of a concentrated aqueous salt solution

The effects of a rapidly heated nanoparticle on the structure of a concentrated aqueous salt solution are studied using molecular dynamics simulations. A diamond-like nanoparticle of radius 20 Å is immersed in a sodium-chloride solution at 20% above the experimental saturation concentration and equilibrated at T = 293 K and P = 1 atm. The nanoparticle is then rapidly heated to several thousand degrees Kelvin, and the system is held under isobaric-isoenthalpic conditions. It is observed that after 2-3 ns, the salt ions are depleted far more than water molecules from a proximal zone 15-25 Å from the nanoparticle surface. This leads to a transient reduction in molality in the proximal zone and an increase in ion clustering in the distal zone. At longer times, ions begin to diffuse back into the proximal zone. It is speculated that the formation of proximal and distal zones, and the increase in ion clustering, plays a role in the mechanism of nonphotochemical laser-induced nucleation. © 2017 Author(s)

An apodizing phase plate coronagraph for VLT/NACO

We describe a coronagraphic optic for use with CONICA at the VLT that provides suppression of diffraction from 1.8 to 7 lambda/D at 4.05 microns, an optimal wavelength for direct imaging of cool extrasolar planets. The optic is designed to provide 10 magnitudes of contrast at 0.2 arcseconds, over a D-shaped region in the image plane, without the need for any focal plane occulting mask.Comment: 9 pages, 5 figures, to appear in Proc. SPIE Vol. 773

Quantum Monte Carlo calculations of electronic excitation energies: the case of the singlet n→π∗n \to \pi^* (CO) transition in acrolein

We report state-of-the-art quantum Monte Carlo calculations of the singlet $n \to \pi^*$ (CO) vertical excitation energy in the acrolein molecule, extending the recent study of Bouab\c{c}a {\it et al.} [J. Chem. Phys. {\bf 130}, 114107 (2009)]. We investigate the effect of using a Slater basis set instead of a Gaussian basis set, and of using state-average versus state-specific complete-active-space (CAS) wave functions, with or without reoptimization of the coefficients of the configuration state functions (CSFs) and of the orbitals in variational Monte Carlo (VMC). It is found that, with the Slater basis set used here, both state-average and state-specific CAS(6,5) wave functions give an accurate excitation energy in diffusion Monte Carlo (DMC), with or without reoptimization of the CSF and orbital coefficients in the presence of the Jastrow factor. In contrast, the CAS(2,2) wave functions require reoptimization of the CSF and orbital coefficients to give a good DMC excitation energy. Our best estimates of the vertical excitation energy are between 3.86 and 3.89 eV.Comment: 6 pages, 1 figure, 2 tables, to appear in Progress in Theoretical Chemistry and Physic

Quantum Monte Carlo facing the Hartree-Fock symmetry dilemma: The case of hydrogen rings

When using Hartree-Fock (HF) trial wave functions in quantum Monte Carlo calculations, one faces, in case of HF instabilities, the HF symmetry dilemma in choosing between the symmetry-adapted solution of higher HF energy and symmetry-broken solutions of lower HF energies. In this work, we have examined the HF symmetry dilemma in hydrogen rings which present singlet instabilities for sufficiently large rings. We have found that the symmetry-adapted HF wave function gives a lower energy both in variational Monte Carlo and in fixed-node diffusion Monte Carlo. This indicates that the symmetry-adapted wave function has more accurate nodes than the symmetry-broken wave functions, and thus suggests that spatial symmetry is an important criterion for selecting good trial wave functions.Comment: 6 pages, 3 figures, 2 tables, to appear in "Advances in Quantum Monte Carlo", AC